The present invention relates to an information recording medium, such as paper on which so-called multimedia information including audio information, such as speech and music, video information obtainable from a camera, video equipment and the like, digital code data obtainable from a personal computer, a word processor and the like has been recorded as a two-dimensional code pattern which can optically be read, an information recording apparatus for recording the two-dimensional code pattern on the information recording medium and information reproduction system which optically reads the code pattern to reproduce original multimedia information.
Hitherto, a variety of mediums including magnetic tapes and optical disks have been known as mediums for recording speech, music and the like. However, even if a multiplicity of copies of the foregoing mediums are manufactured, the somewhat costly unit price cannot be reduced. Moreover, an excessively large space is required to store the copies. What is worse, in a case where the medium, on which speech has been recorded must be sent to a remote person, labor and time takes wastefully even if it is mailed or directly transported. Also so-called multimedia information including video information obtainable from camera, video equipment and the like and digital code data obtainable from a personal computer, word processor and the like except audio information encounters similar problems.
To solve the above-mentioned problems, the applicant of the present invention has, in EP 0,670,555A1 (corresponding to U.S. Ser. No. 08/407,018) disclosed a system for recording, on an information recording medium such as paper, multimedia information including at least any one of audio information, video information and digital code data as image information which can be facsimiled and which can be copied in a large quantity with a low cost, that is, in the form of a two-dimensional code pattern formed by two-dimensionally disposing a plurality of dots serving as encoded information, and a system for reproducing the two-dimensional code pattern.
The two-dimensional code pattern disclosed above is formed as shown in FIG. 1. That is, FIGS. 1A-1C correspond to FIG. 16 of EP 0,670,555A1 and shows a dot code 170 (see FIG. 1A) serving as a two-dimensional code pattern. Area A of FIG. 1 is shown in FIG. 1B the data format of the dot code 170 is structured such that one block 172 (see FIG. 1C ) comprise a marker 174, a block address 176, an error detection and error correction data 178 and a data area 180 in which actual data is set. The blocks 172 are two-dimensionally arranged in the longitudinal and lateral directions to form a dot code 170.
FIG. 2 corresponds to FIG. 17 of EP 0,670,555A1 and shows the structure of an apparatus for reproducing multimedia information. The information reproducing apparatus comprises a detection section 184 for reading a dot code from a sheet 182 on which the dot code 170 is printed, a scan conversion section 186 which recognizes image data supplied from the detection section 184 as a dot code to normalize image data above, a binarizing section 188 for binarizing multivalue data, a demodulating section 190, an adjustment section 192 for adjusting data row, a data error correction section 194 for correcting a read error when data is reproduced and a data error, a data separation section 196 for separating data in accordance with the attitude of data, a decompression processing section for decompressing data compressed to be adaptable to the attitude of data, a display section or a reproducing section or another input device.
In the detection section 184, the dot code 170 on the sheet 182 is irradiated with light emitted from a light source 198, and reflected light is allowed to pass through an image forming optical system 200 having lenses and the like and a spatial filter 202 arranged to, for example, remove moire and the like, and then supplied to an image pickup section 204 arranged to convert optical information into an electric signal and comprising, for example, a CCD or a CMD so as to be detected as an image signal. The image signal is amplified by a preamplifier 206. The light source 198, image forming optical system 200, spatial filter 202, image pickup section 204 and the preamplifier 206 are accommodated in an external light insulating section 208 for insulating disturbance caused by external light. The image signal amplified by the preamplifier 206 is converted into digital information by an A/D conversion section 210, and then supplied to the following scan conversion section 186.
Note that the image pickup section 204 is controlled by a image pickup section control section 212. In an example case where an inter-line type CCD is employed to serve as the image pickup section 204, the image pickup section control section 212 transmits control signals including a V blank signal for vertical synchronization, an image pickup device reset pulse signal for resetting information charge, a charge transfer gate pulse signal for transmitting charges stored in charge transfer storage sections arranged two-dimensionally, a horizontal charge transfer CLK signal which is a transfer clock signal for a horizontal shift register for transferring a charge in the horizontal direction to transmit the same to the outside, a vertical charge transfer pulse signal for transferring the plurality of vertical shift register charges into a vertical direction to transfer the same to the horizontal shift register and the like.
The image pickup section control section 212 supplies a light emission cell control pulse to keep light emission timing of the light source 198 in synchronization with the timing of the transmission of the signals.
Image data is read in a period from a V blank to another V blank in one field. The light source 198 does not continuously emit light but it emits light pulses with synchronization in field units. To prevent introduction of clock noise into the output is when the pulse is emitted, timing is controlled in such a manner that exposure is performed during the V blank period, that is, a period in which the image charge is not transmitted. That is, the light emission cell control pulse is a very thin digital clock pulse which is generated instantaneously and thus supplies great electric power to the power source. Therefore, any contrivance is required to prevent unintentional introduction of noise into the analog image signal. Thus, the light source is caused to emit pulse light beams in the V blank period. As a result, S/N can be improved. The pulse light emission causing the time in which light is emitted to be shortened is significantly effective in eliminating influence of shake occurring when a manual scanning operation is performed and out of focus caused from movement of the image pickup section 204. As a result, a high speed scanning operation can be performed.
In order to prevent deterioration in the S/N ratio because of disturbance, such as external light, in spite of existence of the external light insulating section 208 when, for example, the reproducing apparatus has been inclined, a pulse for resetting the image pickup device is transmitted immediately before the light source 198 emits light in the V blank period to reset the signal of the image. Immediately after the signal has been reset, light is emitted, and immediately after the light emission, image data is read.
The scan conversion section 186 will now be described. The scan conversion section 186 is a section which recognizes image data supplied from the detection section 184 as a dot code so as to normalize the dot code. The recognition and normalization are performed as follows: initially, image data supplied from the detection section 184 is stored in an image memory 214, and then image data above is read out and supplied to a marker detection section 216. The marker detection section 216 detects a marker of each block. A data arrangement direction detection section 218 uses the markers to detect rotation or inclination or data arrangement direction. In accordance with a result of the detection, an address control section 220 reads image data from the image memory 214 to perform correction so as to supply image data to an interpolating section 222. At this time, distortion of each lens occurring in the image forming optical system 200 of the detection section 184 is corrected in accordance with information about the aberration of the lenses supplied from the correcting memory 224 so as to as well as correct the lenses. Then, the interpolating section 222 subjects image data to an interpolating process so as to convert image data into an original dot code pattern.
An output from the interpolating section 222 is supplied to the binarizing section 188. Since the dot code 170 is basically composed of white and black patterns, that is, it is binary information, the dot code 170 is binarized by the binarizing section 188. At this time, the binarizing operation is appropriately performed while causing a threshold value determination circuit 226 to determine the threshold value in consideration of an influence of disturbance and an influence of the amplitude of the signal.
Since modulation has been performed when data has been recorded, demodulation of data is initially performed by the demodulating section 190, and then demodulated data is supplied to the data row adjustment section 192.
The data row adjustment section 192 initially causes a block address detection section 228 to detect the block address of the two-dimensional block. Then, a block address error detection and correction section 230 detects and corrects an error of the block address, and then an address control section 232, in block units, stores data in a data memory section 234. Since data is stored in block units as described above, data can be stored efficiently even if intermediate data does not exist or data is input afterwards.
Then, data read from the data memory section 234 is supplied to the data error correction section 194 so that data errors are corrected. An output from the data error correction section 194 is branched into two sections one of which is, through an I/F 236, supplied to a personal computer, a word processor, an electronic notebook or the like. The other output is supplied to the data separation section 196 so that data is divided into images, characters written by the hand, graphs, characters, line drawings and sound (two types of which is original sound and synthesized sound).
The image corresponds to a natural image and is a multivalue image. The image is decompressed by a decompression processing section 238 to correspond to a compression method, for example, JPEG, and then data, which cannot be corrected, is interpolated by a data interpolating circuit 240.
Binary image information, such as characters written by the hand, graphs and the like, are decompressed by a decompression processing section 242 to correspond to the compression method, for example, MR, MH or MMR, and then data, which cannot be corrected, is interpolated by a data interpolating circuit 244.
Characters and line drawings are, by a PDL (Page Description Language) processing section 246, converted into another pattern for display. Also line drawings and characters, which have been encoded and decompressed for the code, are decompressed (for example, a Huffman coding, Lempel-Ziv coding or the like) by a decompression processing section 248 corresponding to the compression method, and then supplied to the PDL processing section 246.
Outputs from the data interpolating circuits 240 and 244 and the PDL processing section 246 are, by a synthesizing or switch circuit 250, synthesized or selected, and then converted into an analog signal by a D/A conversion section 252 so as to be displayed on a display unit 254, such as a CRT (a TV monitor) or an FMD (Face Mounted Display). Note that the FMD is a glasses-type monitor (handy monitor) to be mounted on the face of the user, and can be effectively used for, e.g., a virtual reality operation or looking at an image on a large frame in a narrow space.
Speech information is decompressed by a decompression section 256 to correspond to the ADPCM coding or the like, and data, the error of which cannot be corrected, is interpolated by a data interpolating circuit 258. In a case of speech synthesis, a speech synthesizing section 260 is supplied with the code of the speech synthesis to synthesize speech from the code to transmit the speech. In a case where the code is compressed in the foregoing case, the decompression processing section 262 subjects the code to a decompression process, such as Huffman coding, Lempel-Ziv coding or the like, similarly to the characters and the line drawings, and then the speech synthesis is performed.
Outputs from the data interpolating circuit 258 and the speech synthesizing section 260 are, by a synthesizing or switch circuit 264, synthesized or selected so as to be converted into an analog signal by D/A conversion section 266, and then transmitted to a speech output unit 268, such as a loudspeaker, a headphone or the like.
Characters and line drawings are directly transmitted from the data separation section 196 to a page printer or a plotter 270 so as to be printed on a paper sheet as characters of a word processor, or enabled to be transmitted to the plotter as a line drawing or a drawing.
As a matter of course, also the image can be printed by a video printer as well as being displayed on the CRT and the FMD, and moreover the image can be photographed.
An image reproducing apparatus having the foregoing structure is arranged such that, for example, the detection section 184 and the scan conversion section 186 are accommodated in a pen-like case so as to be used as a reading section for optically reading the dot code 170 on the sheet 182; and the reading section is held by the hand to manually scan the surface of the sheet 182 along the recorded dot code 170 so as to read the code.
A group including the inventor of the present invention has developed a format which is capable of raising the recording density of dot codes of the foregoing type and has filed in U.S. Ser. No. 08/571,776 (corresponding to EP 0,717,398A3). The format of the dot code is, as shown in FIG. 3, structured such that predetermined matching pattern dots 278 are disposed at predetermined positions with respect to the marker 174, for example, between markers adjacent in the first direction. Moreover, address dots 280 indicating the address of the block is disposed at a predetermined position, for example, between markers adjacent in a second direction. Each of the matching pattern dots 278 and the address dots 280 is composed of a dot having the same size as that of the data dots 282 disposed in the data area 180. Thus, the dot code 170 structured as described above enables the direction of the arrangement and the true center of the marker 174 serving as a reference point for reading the data dots 282 to be obtained by using the matching pattern dots 278 having a predetermined pattern. Therefore, the reference points for reading can easily and accurately be obtained. Thus, even if code patterns are recorded densely, the position of each data dot 282 can accurately be calculated so that original multimedia information can reliably be reproduced.
As the binarizing section 188 for appropriately performing the binarizing operation while causing threshold value determining circuit 266 to determine the threshold value, a binarizing circuit of a type disclosed in Jpn. Pat. Appln. KOKAI No. 59-61383 has been known. The foregoing binarizing circuit is arranged to obtain the maximum and minimum value of digital data converted by an A/D converter, the maximum and minimum value being those in the previous frame. In accordance with the obtained values, a threshold value is calculated and the obtained result of the calculation is used as the threshold value of the present frame so as to perform the binarizing operation.
The above-mentioned system for reproducing the dot code 170 is considerably affected by direct reflection because of the short distance from the surface of the sheet 182 to the image pickup section 204. Thus, a video signal obtained from the image pickup section 204 unintentionally includes noise. If the image pickup section 204 has a defective pixels, noise is generated. Noise of the foregoing type results in that the maximum and minimum values cannot accurately be obtained by the binarizing circuit. As a result, a binarizing operation using an appropriate threshold value cannot be performed. Even if an appropriate value can be obtained, an optimum threshold value cannot always be obtained from the maximum and minimum values attributable to the state where dots are printed.